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This  Keynote  speech  will  actually  address  two  focal  topics:  (1)  certain  suggestions  for  extensions  and 
modifications  of  the  well-known  “JDL”Data  Fusion  process  model,  and  (2)  perspectives  on  an  overarching 
approach  toward  developing  first  Theories  and  then  Algorithms  for  Level  2  Data  Fusion. 


1.0  EXTENSIONS  AND  MODIFICATIONS  TO  THE  JDL  DATA  FUSION  MODEL 

The  JDL  Data  Fusion  process  model  has  certainly  served  the  Data  Fusion  (DF)  community  well  as  a  means 
for  describing  and  understanding  the  nature  of  the  DF  information  process.  It  has  not  however  been  recently 
reviewed  as  regards  application  to  the  framework  of  a  distributed  information  and  fusion  environment. 
Distributed  Data  Fusion  (DDF)  will  no  doubt  form  one  important  undeipinning  for  the  realization  of  Network- 
Centric  Warfare,  and  it  is  in  this  motivational  context  that  the  preliminary  ideas  herein  are  offered.  Certain 
other  ideas  involving  some  suggested  modifications  of  the  model  are  also  offered.  The  (incomplete) 
suggestions  are  shown  in  Slide  4  in  the  Keynote  presentation.  There  are  two  categories  of  suggestions  as  just 
remarked,  one  for  the  DDF  condition  and  one  regarding  new  thoughts  for  an  improved  specification  and 
characterization  of  the  DF  process. 

1.1  Extensions  Regarding  DDF 

Four  major  extensions  are  suggested:  one  related  to  the  “Pedigree”  function,  one  related  to  an  “Adjudication” 
function,  one  related  to  the  need  for  both  local  fusion  and  network-based  fusion  algorithms,  and  one  related  to 
the  sharing  of  information  across  the  nodes  of  the  network.  The  Pedigree  function  is  defined  as  comprising 
those  sub-functions  and  items  of  information  necessary  to  maintain  formal  mathematical  (or,  generally, 
algorithmic)  integrity  for  fusion  of  the  received  information  at  the  receiving  node.  The  Adjudication  function 
comprises  those  sub-functions  necessary  to  address  intemodal  conflict  in  fused  state  estimates  (eg  as  required 
to  arbitrate  and  evolve  the  “Common  Operating  Picture”).  These  functions  are  shown  notionally  in  Slide  4  as 
they  would  be  involved  in  intemodal  processing  in  a  DDF  architecture.  Note  that  we  see  the  Pedigree 
function  as  necessary  for  both  the  intemodal  exchange  of  either  measurements  or  fused  state  estimates, 
whereas  the  Adjudication  function  is  presumed  only  applicable  to  the  exchange  of  fused  estimates. 
We  suggest  making  obvious  the  fact  that  in  a  netted  environment  there  will  typically  be  a  need  for  two  classes 
of  fusion  processing,  one  for  local  fusion  (eg  of  node-organic  sensor  data),  and  one  to  fuse  the  local 
information  or  estimates  with  that  received  from  the  network.  Finally,  we  characterize  the  overall  processes 
involved  with  intemodal  sharing  of  data  or  information  as  “Information  Sharing  Strategies  or  ISS”;  these 
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functions  contain  whatever  logic  necessary  to  govern  the  protocols  by  which  nodes  decide  to  send  particular 
information  to  other  nodes  or  to  broadcast  it,  etc.  That  is,  the  ISS  is  the  logic  which  determines  “who  gets 
what  information,  how  rapidly,  in  what  form,  etc”. 

1.2  Modifications  of  Basic  DF  Processing 

We  also  see  a  deficiency  in  the  current  JDL  model  characterization  regarding  the  role  of  quality  control, 
and  inter-level  processing.  A  frequent  criticism  of  the  model  is  that  it  appears  to  hint  at  sequential  processing; 
this  of  course  was  never  intended  but  admittedly  does  come  across  in  the  diagrammatic  construct  of  the 
model.  We  suggest  to  repair  this  by  making  obvious  the  need  for  intemodal  bilevel  processing,  as  shown  in 
Slide  4.  We  also  envision  a  two-step  qulaity  control  process  before  a  node  issues  an  estimate:  one  that  checks 
whether  a  just-formed  estimate  is  of  adequate  quality  (if  not  a  Level  4  directive  for  additional  information 
might  be  issued,  for  example),  and  what  we  label  as  a  “Belief  Revision”process  that  checks  for  both  intemodal 
consistency  and  consistency  with  recent  global  estimates. 

Lastly,  we  suggest  that  the  basic  DF  model  be  modified  to  make  obvious  the  co-processing  of  both 
inductively-based  and  deductively-based  inferencing  processes.  This  suggestion  is  driven  by  the  new  need  for 
addressing  problems  in  which  the  degree  of  a  priori  knowledge  about  adversarial  behavior  is  very  limited  such 
as  terrorism  problems  and  the  variety  of  asymmetric  warfare  problems.  We  are  not  the  first  to  suggest  this;  as 
noted  in  Slide  6,  Waltz  has  previously  suggested  the  need  for  co-processing  of  Data  Mining  and  Data  Fusion 
operations. 


2.0  PERSPECTIVES  ON  AN  OVERARCHING  APPROACH  TOWARD 
DEVELOPING  FIRST  THEORIES  AND  THEN  ALGORITHMS  FOR 
LEVEL  2  DATA  FUSION 

The  views  that  are  depicted  in  Slides  9  onward  offer  some  ideas  for  moving  forward  in  framing  a  formalized 
approach  to  dealing  with  Level  2  fusion  problems.  The  suggestions  revolve  initially  about  the  assertion  that 
part  of  the  fusion  community’s  problem  in  making  headway  in  both  scientific  understanding  of  Level  2 
concepts,  in  development  of  Theories  related  to  Level  2,  in  algorithmic  specification,  and  then  application- 
oriented  accomplishments  is  that  we  have  not  been  adequately  specific  in  the  characterization  of  the  elements 
of  “situational  constructs”.  Slide  1 1  argues  this  point,  in  noting  that  the  terms  shown  there  are  (more  or  less) 
as  specific  as  we  have  been  in  describing  what  Level  2  is  about.  The  need  for  specificity  is  thought  to  follow 
directly  from  the  fact  that,  if  we  are  to  estimate  Level  2  states  of  interest,  we  must  define  what  those  states  are. 
In  the  end,  we  see  this  as  a  need  for  an  Ontological  specification  of  Level  2  state  elements.  An  ontological 
specification  is  in  itself  an  Existential  Theory,  as  it  is  descriptive  of  how  we  see  the  real  (truth)  world  for  any 
Level  2  application.  We  then  remark  on  the  qualites  of  a  Theory  (Slides  17,18)  and,  drawn  from  the 
Ontological  literature,  suggest  how  Ontology  may  help  in  the  framing  of  necessary  theories  (Slide  19).  It  is 
suggested  in  Slide  20  that  progress  has  been  made  at  Level  1  without  formalized  ontological  analysis  because 
the  real-world  concepts  are  adequately  specific  and  adequately  well-understood  such  that  engineering  theories 
and  algorithms  can  be  nominated  to  form  desired  state  estimates.  In  Slide  21  we  argue  that  that  is  not  the  case 
for  Level  2.  The  next  Slides  through  Slide  30  discuss  the  important  interaction  (yet  to  be  better  or  well- 
understood)  between  formalized  processes  by  which  application  and  user-specific  information  needs  are 
defined  (such  as  by  Cognitive  Work  Analysis,  CWA),  and  the  formalisms  of  Ontological  specification.  These 
two  related  but  not  equivalent  approaches  to  detailed  specification  of  the  information  elements  of  a  Level  2 
conceptualization  must  be  harmonized  in  some  way;  ie  we  must  be  able  to  relate  “user  language”to 
“ontological  language”.  Slides  3 1  and  32  simply  depict  the  concern  that  if  we  must  develop  mission-specific 
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ontologies  (as  the  ontological  community  suggests  that  practical  ontologies  are  of  course  application-specific), 
then  there  is  a  scope  problem  of  constructing  a  wide  base  of  ontologies. 

Slides  33  onward  to  Slide  52  review  a  variety  of  works  in  the  literature  that  specifically  address  various  of  the 
basic  notions  we  have  so  far  (withoput  a  proper  ontology)  ascribed  to  Level  2;  these  slides  point  out  that  there 
is  a  body  of  theoretical  literature  that  exists  that  in  fact  attempts,  albeit  not  in  the  defense  applications  of 
interest  here,  to  offer  various  fonnalisms  to  deal  with  situational  states,  behaviors  of  individuals  and  groups, 
with  human  intentionality,  etc.  It  is  suggested  and  remarked  that  the  fusion  community  has  apparently  not 
drawn  possibly-relevant  ideas  for  its  own  purposes  from  these  possibly-helpful  theoretical  foundations. 
Summarizing  in  Slide  53,  we  suggest  that,  following  the  formation  of  a  relevant  ontology,  there  would  be 
suggested  various  relevant  theories  for  describing  the  relationships  among  situational  components  and  their 
attributes  that  can  then  be  used  as  a  basis  for  eventual  algorithm  development.  Earlier,  in  Slides  20  and  21, 
we  have  asserted  the  distinction  between  Theories  and  Algorithms  in  that  theories  are  descriptive  of  the  real 
world  in  a  context  that  is  unconstrained  by  limits  of  observability,  whereas  algorithms  are  in  essence 
approximations  to  the  theories  governed  by  or  constrained  by  particular  observational  limitations. 
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Discussion  Topics 


•  Remarks  on  the  Fusion  Process  Model 

•  Remarks  on  Level  2  Information  Fusion 
Theories 
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Data  Fusion  Process  Model 


DATA  FUSION  DOMAIN 


•  Intel  Sources 
Air  Surveillance 
•  Surface 
Surveillance 

Spae$^S§£illance 

'g&mp 


Level  0 
Processing 

Sub-object  Data 
Association  & 
Estimation 


Level  1 
Processing 

Object 

Refinement 


Level  4 
Processing 

Process 

Refinement 


Level  2 
Processing 

Situation 

Refinement 


Level  3 
Processing 

Impact 

Assessment 


Data  Base 

Management  System 


Human 
Computer 
Interaction , 


Level  5 


Level  0  —  Sub-Object  Data  Association  &  Estimation:  pixel/signal  level  data  association  and  characterization 

Level  1  —  Object  Refinement:  observation-to-track  association,  continuous  state  estimation  (e.g.  kinematics)  and  discrete  state 
estimation  (e.g.  target  type  and  ID)  and  prediction 


Level  2  —  Situation  Refinement:  object  clustering  and  relational  analysis,  to  include  force  structure  and  cross  force  relations, 
communications,  physical  context,  etc 


Level  3  —  Impact  Assessment:  [Threat  Refinement):  threat  intent  estimation,  revent  prediction),  consequence  prediction, 
susceptibility  and  vulnerability  assessment 


Level  4:  Process  Refinement:  adaptive  search  and  processing  (an  element  of  resource  management) 
Level  5:  Human-Fusion  Process  Interface 


KN2-3 
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Network  Processes 
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Extensions  to  the  Distributed  Case 


•  New  Components  of  the  Model: 

-  Formalization,  at  least  conceptually,  of  the  nature  of  the  Inter-Level 
processes,  and  subsequent  Within-Level  Exploitation  and  Fusion 
processes 

-  Formalization  of  the  means  by  which  overall  (multi-Level)  Process 
Quality  and  Consistency  is  maintained 

-  Formalization  of  the  conceptual  nature  of  both  Adjudication  and 
Pedigree  processes  for  Distributed  Data  Fusion 

-  Formalization  of  the  nature  of  internodal  Information-Sharing 
Strategies 

-  Specification  of  Node-specific  and  Inter-Nodal  Algorithm  pairings 

•  A  Data  Fusion  Process  Ontology 
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One  Further  Thought: 

Combining  Inductive/Abductive  &  Deductive  Processes* 


Table  2  -  Comparison  of  Data  Fusion  and  Data  Mining  Processes 


Process: 

DATA  MINING 

DATA  FUSION 

Knowledge 

Created 

Discovery  of  the  existence  of  previously 
unrecognized  patterns  associated  with  entities  or 
events  in  lime  or  space 

Detection  of  the  presence  of  known  entity  or  event 
types  in  time  or  space  applying  previously  learned 
models 

Input  Data 

Pre- Operational-  Statistically  large  collection  of 
representative  data  containing  observations  of 
known  targets  with  ground  truth 

Operational  Data  collected  by  multiple  sensors 
and  sources 

Reasoning 

Process 

Abduction-Induction.  Discovery  of  sufficient 
correlated  relationships  in  data  to  infer  a  genera! 
description  (or  rule  set)  that  may  be  always  or 
generally  (to  some  quantified  degree)  true 

Deduction:  Detection  of  previously  known  patterns 
in  data  to  infer  the  presence  and  identity  of  the 
entity  or  event  represented  by  that  pattern 

Knowledge 
Patterns 
(Models)  Used 

Unknown:  (general)  model  of  uinteresting”  data 
properties  is  used  as  template  to  detect  qualifying 
candidates  for  "new"  models  of  targets  of  interest 

Known: (specific)  models  arc  used  as  templates  to 
detect  similar  patterns  in  data. 

Object  of 
Detection/ 
Discovery 
Process 

and 

Knowledge 

Gained 

•  Discovery  of  "interesting"  general 
relationships  and  patterns  of  behavior, 
which  may  be.  validated  as  general  models 
of  relationships  or  behavior. 

•  Discovery  of  new  types  of  entities  or 
events,  by  previously  unidentified  and 
unknown  patterns,  in  large  volumes  of  data 

•  Detection  of  individual  and  related  sets  of 
entities  and  events. 

•  Detection  of  the  presence,  type  and  location 
of  known  types  of  entities  or  events  in  large 
volumes  of  data 

Output 

Knowledge 

Models  of  relationships  or  behaviors  to  describe 
target  entities  or  events 

Detected  targets  based  on  matching  sensor  data 
with  known  models  of  entities  or  events 

*  Waltz,  E.,  “Information  Understanding:  Integrating  Data  Fusion  and  Data  Mining  Processes”,  1998 
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One  Thought  on  a  Composite  Architecture 
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Additional  Issues: 

Incorporation  of  learned  models  into  Deductive  side 
•  Pattern  persistence  vs  mission  urgency 
•  Retrospective  processing 
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*  Waltz,  E.,  “Information  Understanding:  Integrating  Data  Fusion  and  Data  Mining  Processes”,  1998 
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Combining  Inductive/ Abductive  &  Deductive  Processes 

•New  Components  of  the  Model  Related  to 
Inductive/ Abductive  Processes: 

-  “LO”:  Data  preparation,  cleansing,  alignment 

-  “LI”:  Data  Mining  Pattern  Search 

-  “L2”:  Knowledge/Model  Discovery 

-  Model  Quality  Control 

•  Qualification  metrics 

-  Model  Integration 

•  Issue:  Human  Role  in  this  process 
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Scientific  Foundations  of  Level  2 

Fusion 
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What  is  Level  2  Fusion  About? 


“Situation  Estimation” 


Estimates  about  Relationships  among 
and  Activities  involving  Physical  and  Non-physical  “Objects” 
in  the  Real-World  as  Observed  by  the  System  at  Hand 


“Events  and  Behaviors” 


“Aggregation” 
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The  JDL  Historical  View  of  the  Nature  of  Fusion 

Products  across  Levels 

(R.  Antony,  circa  1989) 
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An  Assertion 


•  The  lack  of  specificity  and  formal  analysis 
in  these  characterizations  prevents 
mounting  a  consistent,  scientifically- 
grounded  approach  to  L2  algorithm 
development 
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The  Starting  Point  for  Algorithm 

Development: 


Understanding  the  Real-World: 
The  need  for  a  L2  Ontology 
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Why  should  the  DF  Community  be  Concerned 

with  Ontology? 


•  If  we  are  to  have  Theories  about  and  (later)  Algorithms 
to  estimate  “States”  of  interest  about  objects  and 
activities  in  a  problem  space,  we  must  be  able  to 
describe,  represent,  and  specify  those  States  in  the  Real 
World  in  some  orderly  way 

•  It  is  the  same  problem  the  community  faced  early  on, 
when  developing  an  initial  Taxonomy 

•  It  is  what  allows  us  to  understand  what  is  meant  by  the 
“Level  1”  terms: 

-  Tracks,  maneuvers,  classes/types  of  object  identity,  “move- 
stop-move  behaviors,  intersensor  bias,  air  corridor,  etc 

•  Not  as  estimated  from  noisy  observables  but  the  concepts/notions  as 

exist  in  truth  (at  least  initially) 
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Value  of  an  Existential  Theory  * 


“A  sound  and  complete  (existential)  theory  E  will  ensure 
that 

inconsistencies  and  ignorance  in  our  data  fusion  system 

derive  from  our  system’s  knowledge  of  the  world,  and 
not 

from  the  meaning  of  the  terms  by  which  the  system 
describes  the  world,  (ie  our  existential  ontology)” 


*  Lambert,  D.,  “Grand  Challenges  of  Information  Fusion”,  FUSION2003,  Cairns,  Australia,  July  2003 
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Theories,  Algorithms,  Models,  and 

Ontology 
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On  Theories  * 


•  Nature  of  a  Theory 

-  A  statement  about  relationships 

-  An  explanation  of  a  phenomenon 

-  Possibly  a  statement  about  constraints  or  limiting 
conditions 

•  Criteria  for  a  Theory 

-  Verifiability  thru  experimentation  (Falsifiability) 

-  Internal  consistency,  coherence 

-  Parsimonious 

-  Explanatory  capability 


*  K.  Shaver,  “Validity  in  Research”,  Lecture  at  College  of  William  &  Mary 
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Inconsistency  in  Theories  * 


•  “There  are  two  main  types  of  inconsistency  concerning  a  theory  in 
physical  sciences:  Formal  inconsistency,  and  factual  inconsistency. 


•  Formal  inconsistency  arises  when  the  theory  has  two  or  more 
hypotheses  that  contradict  one  another.  Formal  inconsistencies  can  be 
divided  in  two  categories:  Logico-mathematical,  and  conceptual. 


Logical  inconsistencies  are  easier  to  detect  and  are  usually  eliminated  in 
the  early  development  of  a  theory  by  logical  operations  (e.g.  by  deleting 
one  of  the  contradicting  hypotheses). 


-  Conceptual  contradictions  are  more  difficult  to  identify  and  eliminate . 
They  come  into  notice  in  the  later  stages  of  theory  development.  “ 


*  Kocabas,  S.,  Symp.  On  Computational  Discovery  of  Communicable  Knowledge,  Stanford  Univ,  Mar.,  2001 
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Theory  Development  and  Ontology: 
How  does  Ontology  help? 


•  Clarity  in  describing  the  real-world 

•  Consistent,  shareable  descriptive  framework 

•  Organized  framework  for  describing 
relationships — axiomatically-based 
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_ What  about  a  LI  Ontology? 

•  The  fusion  community  has  succeeded  without  a  (formal) 
LI  ontology  because  the  concepts,  as  we  have 
characterized  them,  are  : 

-  sufficiently  precise  to  engineer  to  in  the  sense  of 
algorithms  and  other  engineering  aspects  (eg 
visualization) 

-  (relatively)  well-understood  in  common  use — thus  no 
major  consensus  issue 

J  Observability 


Single-object 
Tracking 
Algorithm 
Sufficient  for: 
-application  needs 
-handling  observ. 
qualities 
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Things  in  the 
Real  World 


Partitioning 

And 

Labeling 


(Analysis ;  Ontology — 
Sufficient  Specificity 
To  develop  Theories) 


iflrrnrari  Mirer 


ifl  nr. raw  Mire: 


ii'iii  ■iium  i'j  in 


!g<oin*inira 


Nature  of  Real  3D 
Single-object 
Dynamic  motion, 
maneuverability 

Real 
World 


Model  (Algorithm)  re 


Model  f Algorithm!  re 

Model  (Algorithm)  re 
Thing-component 
In  the 

Observed  World 


Assumptions, 

Approximations, 

Application-needs 

Observed  World 

in  the  Application  (Task)  Conte? 


Algorithm  Development 


•  Algorithm  development  is  a  process  that  has  its  foundation  first  in 
a  mentally-grounded  or  scientifically-grounded  Theory  and  the 
associated  descriptive  language  about  how  things  are— in  truth— in 
the  Real  World 

•  It  is  about  formulating  a  numerical  or  symbolic  approach  to 
estimating  or  representing  some  features/some  relationship  about 
things  in  the  Real  World  (an  approximation  to  or  model  of) — or  to 
employ  a  Theory  about  things  or  concepts  in  the  Real  World 

•  Later  in  development  of  an  Algorithm,  we  must  deal  with: 

-  Imperfect/flawed  observations 

-  Other  application-specific  factors 

-  Implementation  in  a  cost-effective,  efficient,  computational 
framework 

-  etc 
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More  on  Models  and  Modeling  * 

•  Models  are  (idealized)  analogs  of  the  phenomena  that  they  represent. 

•  “Models  are  not  literal  copies  of  that  which  they  represent.  Rather,  they 
are  conceptual  tools  for  selecting  and  organizing  salient  features  such 

as  structural  elements  and  their  relationships ,  processes ,  and 
functions” 

-  Ontologies  give  us  a  framework  for  such  “salient  features” 

•  “Scientists  construct  and  use  models  for  a  variety  of  purposes:  a)  to 
make  the  “invisible”  overt  or  visible,  b)  to  articulate  the  structure  and 
function  of  their  theories  by  specifying  the  elements  of  models  and 
their  spatial,  temporal,  and  functional  relationships,  c)  to  explain 
phenomena  or  empirical  data,  d)  to  make  predictions,  and  e)  to 
communicate  ideas.” 

-  Models,  built  upon  salient  features  (possibly)  derived  from  Ontologies, 
articulate  theories  of  science 

•  “Thus,  the  ability  to  construct  and  reason  with  models  is  widely 

9g  ancR IQl9fi  n?Bo  scilhce .  Journal  of  Research  in  Science  Teaching, 

28,  73-79.  KN2-23 


Getting  Started 


•  Ok,  if  we  agree  that  we  must  first  address, 
ontologically,  some  range  of  concepts  in  the 
Real  World  somehow  delineated  by  (Informed 
by)  intuitive  notions  of  either  “Situations”  or 
“Threats”,  how  do  we  bound  it?  What’s  “in” 
and  what’s  “out”? 

•  The  problem  of  “Admission” 

-  Must  come  from/be  bounded  by  Missions,  Goals, 
Objectives,  Tasks 

•  To  bound  the  problem 

•  To  focus  the  work  in  a  pragmatic,  utility-oriented  way 
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Another  View  of  Task-driven  Nature  of 

“Situation  Elements”  * 


THE  COMMANDER’S  DECISION-MAKING  METHODOLOGY 

SITUATION  ELEMENTS 


*  Ivanov,  D.A.  et  al,  “Fundamentals  of  Tactical  Command  and  Control:  A  Soviet  View”, 
Soviet  Military  Thought,  No.  18,  translated  and  published  by  USAF,  1977 
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Assessing  User/Task  Information  Needs 
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Layered  Approach  to  Cognitive  Work  Analysis 


*  Reising,  D. ,  “Work  domain  analysis  and  sensors  I:  principles  and  simple  example”,  Int.  J.  Human-Computer  Studies  (2002)  56,  569-596) 
Rasmussen,  J.,  Pejtersen,  A.  M.  &  Goodstein,  L.  P.  (1994).  Cognitive  Systems  Engineering.  New  York:  John  Wiley  &  Sons. 
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Ontological  Construction  Ideally  Derived  from  a 
Synergistic  Relation  with  User  Information  Needs 


1 )  User/T ask  Information  Needs 

INFORM  and  BOUND 
Ontological  Structure. 

•  Top-down  and  Bottom-up 
Construction 

•  Empirically  Driven 

•  Reflects  Epistemically-driven 
(cognitive)  Info  Needs 

•  Possibly  developed  from 
Cognitive  Work  Analysis 
techniques 

•  Sometimes  developed  as  ad 
hoc  Taxonomies,  Dictionaries 


2)  Ontological  Development 
STRUCTURES  and 
VALIDATES  One’s  Common- 
Sense,  Non-formal  Abstractions 
of  the  World. 

•  Generally  Top-down 
Construction 

•  Rationally  Driven 

•  Reflects  Epistemically- 
Independent  Structure  of  the 
Existing  World 


KN2-28 


REAL  WORLD  CONCEPTS 


Info  Needs 
Categories 
(Abstraction  Levels) 


Ontologically-Driven 


Eg,  3700 
Limited  Multi¬ 
mission  EEI’s 
As  per  Keithley 


Study 


Some  Types  of 

“Situation”,  ie  Level  2,  Filters 
—Coarse 
—Fine 


c^Conceptjr^ 


Concept 


CConceptln^ 


Task-Driven 


JDL  Fusion  Model 
Notional  L2  Info 


“Admission”  still  a  Problem 


•  Is  the  ad  hoc  notion  of  “Situation”  (since  the  theories 
on  “situation”  don’t  help  us  much  in  regard  to  our 
applications)  sufficient  as  a  “Coarse  Filter”  to  bound 
what  we  will  attribute  to  Level  2? 

•  How  else  to  do  it? 

•  Reexamine  CCIR  structure  and  partition  to  Fusion 
Levels  in  some  other  way? 

•  We  need  to  specify  “L2  Filters” 
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Scope  of  the  Ontology-Development  Task 


•  Conventional  Threats 

-  Lethality 

-  Intent 

-  Opportunity 

-  Largely  deductive 

•  Asymmetric  Threats 

-  Lethality  =  ? 

-  Intent :  how  to  estimate? 

-  Opportunity:  how  to  estimate? 

-  Deductive  vs  Inductive  approaches 


~?9? 

•  •  • 

Mission 

Areas 

~ 25 

Mission 

Areas 
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_ OQTW _ 

Non-Combatant  Evacuation  Operations 


Arms  Control 

Support  to  Domestic  Civil  Authorities 
Humanitarian  Assistance  and  Disaster  Relief 

Security  Assistance 
Nation  Assistance 

Support  to  Counter-Drug  Operations 
Combating  Terrorism 
Peacekeeping  Operations 
Peace  Enforcement 
Show  of  Force 

Support  for  Insurgencies  and  Counter-Insurgencies 
Attacks  and  Raids 

Asymmetric  Warfare _ 

Weapons  of  Mass 
Destruction  (NBC) 

_  .  3  Mission 

Terrorism ,  Counter- 

Terrorism  Areas 

Information  Warfare 


The  Ontology  Development  Challenge 
for  L2/L3  Fusion 


13  Mission 
Areas 


+  Conventional 
Missions 


Gray  Area  Phenomena 
Ethno-Religious-Nationalistic  Conflict 

Weapons  Proliferation-Conventional  and 
NBC 

Conflict  Over  Scarce  Resources 
Infectious  Disease  Control  Issues 
Globalization  of  Organized  Crime 
Drug  Trafficking 


Economic  Warfare-Conflicts 
Technology 

Emigration  Policies  and  Control 
Famine-Related 


Over 


9  Mission 
Areas 
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Other  Aspects  of  L2  Scientific 

Foundations 
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How  is  Data  Fusion  Done? 

—Extensions  of  conventional  (single-source)  methods 
—New,  creative  techniques  specially  designed  for  IF 


•  Intel  Sources 
•  Air  Surveillance 
Surface  Surveillance 
Space  Surveillance 


LO:  Sub-Object 
Association/ 
Correlation 

•Combinatoric 

Optimization 

•Numerical/Statistical 
Estimation  Techniques 


Association/ 

Correlation 

•Combinatoric 

Optimization 

Object  Refinement 

•Numerical/Statistical 
Estimation  Technique 

•Pattern  Recogni) 
Techniques 


L2:  Situation  Refinement 

•  Numerical/Statistical 
Estimation  Techniques 

•  Knowledge-based, 
Symbolic  Techniques 


L3:  Impact  Assessment 

Numerical/Statistical 
Estimation  Techniques 

Knowledge-based, 
Symbolic  Techniques 


L4:  AdapWr 


Not  very  specific 
Vague  in  defining  scientific  foundations 


UUI  III  VJI  JU  IJUI 


Source  Management;  Info-theoretic  Techniques 
Process  Adaptation;  Control-theoretic  Techniques 
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Terrain  &  prior 
intelligence 


Scientific  Foundations: 
Possibly-Relevant  Theories  for  L2 


•  Ontology  is  one  of  them — the  needed  Existential  Theory 

•  What  else? 

-  Theories  of  Situations 

•  Situational  Theory 

•  Situation  Semantics 

•  Situational  Calculus 

-  Theories  of  Behaviors  (and  Tactics  as  appropriate) 

-  Theories  about  Intent 

-  Theories  about  Aggregations 

-  Theories  about  Context 

-  Theories  about  Relationships  as  defined  in  the  Ontology 
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Searching  the  Literature: 


Scientific  methods  for,  Theories  about 

“Situations” 
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Classifying  Situations  * 


•  Vendler  classified  situations  into  four 
classes: 

-  Activities:  run,  push  (a  cart). 

•  Not  time-bounded 

-  Accomplishments:  run  a  mile,  draw  a  circle. 

•  Spans  an  interval  and  has  an  end-point  (telicity;  tending 
toward  an  end-point) 

-  States:  know,  love. 

•  Static;  persist  over  time 

-  Achievements:  find,  recognize,  spot. 

•  spontaneous 


*  Vendler,  Z.  (1957).  “Verbs  and  times”,  The  Philosophical  Review,  66(2):143n160. 

*  Ismail,  H.O.,  “Reasoning  and  Acting  in  Time”,  PhD  Dissertation,  Univ  at  Buffalo,  2001 
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Another  Classification  of  Situations  * 


Situations 


States 


Occurrcn  ces 


Processes 
( Activities  ) 


E 


.tents 


Developments 
(Acc 07 np  Hsh  me  n  ts) 


P  u  n ct  ua l  Occ  urren ces 
(Achievements) 


Mourelatos,  A.  P.  D.  (1978).  “Events,  processes,  and  states”,  Linguistics  and  Philosophy,  2:415n434 
*  Ismail,  H.O.,  “Reasoning  and  Acting  in  Time”,  PhD  Dissertation,  Univ  at  Buffalo,  2001 
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Another  Classification  of  Situations  * 


Situations 


States  Non -Slates 


Dynamic 

Static  Processes  E 

vents 

(Activities)  _ 

- 1 

Protracted 

1 

Momentaneous 

( Accomp  lisk  merits ) 

(Ach  ievements) 

1 

Hupp  on  ings  C  u  bni  nations 


Figure  2,2: 


Bach's  classification  of  situation 


* 


*  Bach,  E.  (1986).  “The  algebra  of  events”,  Linguistics  and  Philosophy,  9:5n16. 
Ismail,  H.O.,  “Reasoning  and  Acting  in  Time”,  PhD  Dissertation,  Univ  at  Buffalo,  2001 
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EVENTS 

STATES 

atomic 

extended 

understand 
love,  know 
resemble 

+conseq 

CULMINATION 

recognize,  spot,  win  the  race 

CULMINATED  PROCESS 

build  a  house,  eat  a  sandwich 

— conseq 

POINT 

hiccup,  tap,  wink 

PROCESS 

run,  swim,  walk,  play  the  piano 

Figure  2.3:  Moens  and  Steedman’s  classification  of  situation  types. 


•Shows  relationship  between  situation-types  via  internals  of  event-structures 
•Events  made  up  of:  preparatory  stage,  a  culmination,  a  consequent  stage 


*  Bach,  E.  (1986).  “The  algebra  of  events”,  Linguistics  and  Philosophy,  9:5n16. 

*  Ismail,  H.O.,  “Reasoning  and  Acting  in  Time”,  PhD  Dissertation,  Univ  at  Buffalo,  2001 
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Dimensions  of  situation-types* 


•  Durative-punctual 

-  Consuming  time  or  not 

•  Telic-atelic 

-  Having  an  end-point  or  not 

•  States  and  non-states 

-  Static,  dynamic 


*  Ismail,  H.O.,  “Reasoning  and  Acting  in  Time”,  PhD  Dissertation,  Univ  at  Buffalo,  2001 
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Situation  Calculus 


•  Situation  calculus: 

-  Refers  to  a  particular  way  of  describing  change  in  first-order  logic 

-  Conceives  of  world  as  consisting  of  a  sequence  of  situations,  each  of  which  is  a 
snapshot  of  the  state  of  the  world 

•  All  relations/properties  of  world  that  can  change  over  time: 

-  Specify  extra  situation  argument  to  predicate 

-  Instead  of:  At(agent,  location) 

-  Indicate:  Atfagent,  location,  Sj),  where  Sj  is  a  particular  situation,  or  point  in  time 

•  Represent  how  world  changes  from  one  situation  to  next: 

-  Result(Forward,  S0)  =  Si 

-  Result(Tum(Right),  S^  =  S2 

-  Results(Forward,  S2)  =  S3 
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Example  of  World  Represented  by  Situation  Calculus 


Science  of  Behaviors 


•  Ethology  is  the  comparative  study  of  behavior 

•  In  a  general  way  it  attempts  to  discover  and 
explain  generalizations  covering  the  behavior  of 
organisms. 

•  However,  Ethological  theory  focus  on  causes  and 
adaptive  value  of  behavior  considering  both 
evolutionary  history  and  social  context  -good  for 
some  problems  (eg  terrorism?)  but  not  all 
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Theories  About  “Behavior”  * 


Suggest  3  components: 

—coordinated  activities 
—collaborative  activities 
—cooperative  activities 


*  Franke,  J.,  et  al”  Enhancing  Teamwork  through  Team-level  Intent  Inference”  LMCO  ATL  Paper, 
http  ://www.  atl. external,  lmco .  com/ overview/papers/997 .pdf 
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Notions  of  “Behavior”  and  “Role” 


•  Behavior:  A  collection  of  actions  with  a  set  of 
constraints  on  when  they  may  occur. . . 

•  Role:  An  abstraction  of  the  behavior  of  an  object 
that  consists  of  a  subset  of  the  interactions  of  that 
object  together  with  a  set  of  constraints  on  when 
they  may  occur.  A  role  always  belongs  to  a 
specific  larger  behavior  that  involves  other  roles, 
called  a  collaborative  behavior. 
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Theories  and  Modeling  of  Intent 


Reasoning 


Culture, 

Value  Systems 


Perception 


Doctrine 


“wish  to  X’ 


Task/Goal 

Assignment 


Current 

Military/Intell 

A  1  * 


Llinas,  J  and  Deutsch,  R.,  “Perspectives  of  Intent  from  Various  Bodies  of  Literature  and  Perspectives  for  Developing 
New  Models  of  Intention  Estimation  And  for  the  Development  of  Decision  Aids”,  CMIF  Report  2-99,  Feb  1999 
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Theories  of  Groups,  Teams,  Coalitions,  etc 

-Adaptive  Behaviors- 

•  There  is  a  huge  literature  on  Group  Behavior;  some 
references: 

-  “Designing  and  Understanding  Adaptive  Group  Behavior”  (1995)  M.  J.  Mataric,  Brandeis  Univ 

-  "Collaborative  Plans  for  Complex  Group  Action."  Grosz,  B.and  Kraus,  S.,  In  Artificial 
Intelligence.  86(2),  pp. 

269-357,  1996 

•  The  Intelligent  Agent  literature  is  perhaps  a  better  basis  for 
researching  ideas  and  concepts  regarding  multi-agent 
behavior  as  it  might  apply  to  Information  Fusion 
applications 

•  Theories: 

-  Teamwork:  see  Tambe,  Cohen  &  Levesque,  etc 

-  Coalitions:  Klusch,  M.,  Gerber,  A.  (2002):  Dynamic  Coalition  Formation 
Among  Rational  Agents.  Journal  IEEE  Intelligent  Systems,  17(3),  IEEE  Press. 
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Theories  and  Notions  of  Context  * 

•  Llinas:  Context  is  an  epistemological  matter,  not  an 
existential  matter 

•  “One  area  almost  totally  devoid  of  systematic 
research  is  that  of  context”. 

•  “Primary  KBS  maintenance  issue  derives  from  lack 
of  contextual-interpretation  capability” 

•  Jansen  reviews  various  literature — all  offering  rather 
vague  characterizations;  most  specific  was: 

-  “Mittal  and  Paris  (1993)  identify  the  following  five  aspects 
of  context,  the  problem  solving  situation;  the 
participants  involved;  the  mode  of  interaction  in  which  the 
communication  is  occurring;  the  discourse  taking  pace; 
and  the  external  world”. 

-  Mittal  VO  &  Paris  CL,  Context:  Identifying  its  Elements  from  the  Communication  Point  of  View,  in  Proceedings  of  the 
IJCAI'93 

*  Jansew^ks^W^ir^  KNOWLEDGE  BASES”, 

CSIRO  Division  of  Information  Technology,  North  Ryde,  NSW  2113,  AUSTRALIA 
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Semantic  Theories  * 

(nature  of  relationships  to  be  studied  at  DSTO;  pursuing  formal  semantic  theories) 

“symbols  capable  of  describing  military  action” 


Social: 


Intentional: 


Functional,: 


Physical: 


group,  ally,  eneiny,  neutral,  own,  possess, 
in  v  i  He,  offer,  ae  cept,  a uthori se,  a  1 1  ow, 
individual,  routine,  learnt,  achieve,  perforn 
succeed,  fail,  intend,  desire,  belief,  expect 
anticipate,  sense,  inform,  effect, approve, 
disapprove,  prefer, 
sense,  move , stri ke,  a ttac h,  i nform , 
opera l i ona I ,  d ism.pt,  iieutra I i se ,  destroy, 
land,  sea,  air,  outer_space,  incline,  decline, 
number,  temperature,  weight,  energy. 


Metaphysical:  exist,  fragment,  identity,  time,  before, 

space,  connect,  distance,  area,  volume, 
angle. 


In  my  view  I  see 
these  as 
helpful  toward 
identifying 

relationship-types — ie 
relationship-types 
within 

each  semantic 
theory-class 

But  I  am 
not  an  Ontologist 


*  D.  Lambert,  DSTO,  FUSION2003 
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Theories  and  Notions  of  Trust 


Trust  Attribute 

Characteristics 

Basis  for  Trust  Development 

Reliability  . 

Repeated,  consistent  functioning 

Conditioning 

Robustness  \ 

Similar  to  Competence;  good 
perfonnance  under  varying  problem 
conditions 

Conditioning;  often  achieved  with  a 
small  sample  of  cases  (and 
potentially  unwarranted) 

Familiarity  \ 

Employment  of 

familiar/friendly/natural  procedures, 
terms,  etc. 

Inherited  from  the  culture  of  the 
domain  (or  life  experience);  can  be 
unwarranted,  irrational 

UnderstandabVty  \ 

Not  the  same  as  Familiarity  or  ease 
of  use;  supports  formation  of  mental 
models  and  predictive  framework 

Some  notion  of  “visibility”  into 
inner  processes.  Distrust  and 
alienation  formed  when  inner 
workings  not  understood. 

Explication  of  IntcntVn  \ 

Not  same  as  Understandability  in 
that  intention  is  overtly  stated', 
Understandability  allows  inferring 
intent 

Execution  of  actions  per  stated 
intent— in  a  sense,  follow-up  on 
intended/stated  actions 

Usefulness  \  \  \ 

Classical  notion  of  utility 

From  use  of  a  “focused”  or 
“directed”  system  that  supports 
assigned  work. 

Dependence  \\\ 

Reliance  on  system  to  support 
human  performance 

Can  be  “Catch-22”  type  relationship- 
-a  type  of forced  trust  on  a  system 
you  (must)  depend  on 

All  are  Design  Factors 


Bisantz,  A.,  et  al,  “Empirical  Investigations  of  Trust-related  System  Vulnerabilities  in  Aided,  Adversarial  Decision 
Making”,  CMIF  Report,  Jan  2000 
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